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ABSTRACT 
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The availability of digital and hybrid computers has led to the 
development of the state space approach and optimization theory for the 
analysis and design of control systems, particularly in space oriented 
problems where meaningful cost criteria can be defined. In this thesis 
optimization theory is investigated as applied to classical control 
systems, such as regulators, to determine if these techniques may be 
used in the design of systems to meet conventional performance standards. 
As part of this investigation a method has been developed which yields 
the overall state equations for system from the state equations of the 

- iJ w 

individual components. Also, since optimal designs are usually non- 
linear and time varying, a discussion oi stability ciiteria xor these 
systems is included. 
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INTRODUCTION 



There are two basic approaches to the analysis and design of 
control systems. The classical approach uses a transfer function to 
describe the system. Since this is primarily a frequency (either s or 
w) domain description, most of the design and analysis techniques are 
formulated in the frequency domain. The state space approach describes 
the system with a set of first order differential equations. Since 
this is primarily a time domain description it is logical that design 
and analysis techniques be formulated in the time domain. Part of the 
analysis problem consists of combining the individual component descrip- 
tions to form the overall system description. The methods for forming 
the system description with the classical transfer function are well 
established. However, for components described in the state space, 
such methods are not established. In Chapter I, a method is introduced 
which permits the engineer to obtain the system equations from the 
component state equations in a straightforward manner. 

One class of design problems which uses the state space description 
is to be found in optimal control theory. Optimization and state space 
techniques can be applied to such problems as minimum time, minimum 
fuel, and trajectory calculations with much success. Their application 
to the design of servomechanisms and regulators is questionable. In 
Chapter II, optimal control theory is applied to the regulator problem 
in order to obtain a better understanding of the meaning of optimality. 
This investigation naturally leads to the question as to how the engineer 
can use this theory to design a regulator to meet specific design 
criteria. 
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Optimal control theory, more often than not, leads to non-linear 
and time varying systems. Stability is therefore an important considera- 
tion when using the state space approach. Since there is no simple 
stability criterion that is applicable to all non-linear and time varying 
systems, Lyapunov’s second method must be used to develop criteria for 
specific classes of non-linearities. Lyapunov’s second method is 
particularly useful in state space because it is formulated in the time 
domain. In Chapter III, stability criteria that exist for a particular 
class of time varying systems are investigated. This system is of 
particular interest because it can be the result of an optimal linear 
regulator design. 
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CHAPTER I 



State Space Approach to Component Interconnections 

1.1. Introduction . Since the transfer function is not a valid representa- 
tion of a non-linear system, design tools such as root locus, Routh 
criterion, Nyquist criterion, and frequency response cannot be applied 
to non-linear systems. The state space description has a wider applica- 
tion because it is a valid representation of non-linear, linear, time 
invariant, and time varying systems. Often in the analysis and design 
problem, the internal behavior of the system is of concern. The transfer 
function is an input - output relationship and therefore provides no 
information about internal behavior. The state equations provide both 
input - output and internal information about the system. The state 
variables themselves determine this internal behavior. It is significant 
to note that the transfer function concept implies a single input-single 
output for each component or block of a system. The state space approach 
enables one to describe the system in a compact and simple form even 
though the system is composed of components which are multi- input and 
multi-output. The objective of this chapter is to present a simple 
method for obtaining the system state equations from the component state 
equations for component interconnections such as cascade, parallel, and 
feedback; and also to present a general method for the same task when 
the system is composed of many components which are interconnected in 
any conceivable way. The advantage of the two methods developed here 

is that the identity of the states of the individual components are 

21 

preserved in the system description. Choate and Sage have developed 
a procedure to yield the system state equations in phase variable form. 
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This is a mathematically convenient form but it does not preserve the 
identity of the individual component states in the system description. 



1.2 State Equations and the Invariance of the Classical Transfer Function . 

Any dynamic component of a control system is completely described 
by the state equations 



x = Ax + e>{L 


(1-D 


Y = C x + D>T 


(1-2) 



where 

A “ an nxn matrix 
B ■ an nxm matrix 
C = an sxn matrix 
D “ an sxm matrix 

A « an nxl column vector of state derivatives 
x = an nXl column vector of states 
/£. “ an mxl input vector of forcing functions 

X “ an sxl output vector 
b *■ the number of outputs 
n " the component order 
m <* the number of inputs 



An expanded form for Equations (1-1) and (1-2) is 
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where A.., B.., C.., and D.. are obtained from the basic differential 
ij i-J ij 

equations describing the component and thus may be constant, time 

varying, or functions of x, or both. Henceforth, Equations (1-3) and 

(1-4) will be referred to as the generic form. 

Equations (1-1) and (1-2) have various special forms for which 

there are unique A, B, C, and D matrices depending upon the choice of 

state variables. In general, the choice of states is arbitrary and 

thus many representations exist. However, only certain specific choices 

have physical meaning (such as measureable indentif iable quantities) in 

any given system. It is worthwhile to demonstrate the invariance of a 

transfer function (single input/single output) to the manner in which 

3 

the states are defined. Let 

Z = TX (1-5) 

Thus, Z is a new set of state variables defined by means of the matrix 
transformation T operating on the given set of states X. Equations 
(1-1) and (1-2) then become 

TZ = A T*Z + B & (1-6) 

Y = CTZ + D n (i-7 ) 

where r and Y are considered as column vectors (multiple input and 
output). Solving for the output, one obtains 
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(1-8) 



Yds) = C t'et's -AT’Tb nio + 0/2 



Equation (1-8) is equivalent to 



Y(^) = ctcf's -AT ')Tj &/2 <s3 + D {l < s > 



= CLsI-Af'b^^) + D (LL^> 



(1-9) 



The output. Equation (1-9), is identical to that obtained from the 
original Equations (1-1) and (1-2). For the case of a single input 
and single output system, the transfer function becomes 



Equation (1-10) is independent of T and is the classical transfer 
function. 

1.2.1 Special Forms . The following special forms are commonly used 
state representations . 

(a) Normal Form. Phase Variables . 
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This form corresponds to the following transfer function for single 
input - s ing le output : 
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(1-13) 



If the transfer function of a plant is expanded in the form of Equation 
(1-13), Equations (1-11) and (1-12) can be written directly provided 

3 

the plant is controllable. The normal form may also be obtained from 

the generic form by performing a matrix transformation such as that 

described by Equation (1-5). The procedure for selecting the T matrix 

that transforms the generic states into Normal Form (phase variables) 
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is discussed by Browne. The procedure is not straightforward and 
requires a great deal of mathematical manipulations. Thus, for a single 
input and single output component, it is usually simpler to go from the 
generic form to the transfer function and then to the Normal Form. This 
particular representation is of interest in Chapter II because it lends 
itself to certain mathematical conveniences not available in the other 
forms . 

(b) Lur'e or Eigenvalue Expansion 
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(1-15) 
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Equations (1-14) and (1-15) correspond to a component with simple poles 
only. For components with repeated poles, the A and B matrices are 
altered.^ - This Lur'e form corresponds to the following transfer function 
(for the component with simple poles only): 



D„ + -£u_ + -JBu- + 
S-A„ ^ S-A zz + 



-&.IM 

S "Aon 



(l-15a) 



This state variable representation is obtained from the partial fraction 
expansion of the transfer function. However, it may also be obtained 
from the generic form by a proper choice of the T matrix. ^ 

In practice, if the differential equation relating input to output 
is available, then the transfer function is easily obtained and the 
Normal form or Lur*e expansion follow readily. However, if the system 
is described by a sequence of differential equations then the generic 
form may be obtained easily. For multiple input and multiple output 
systems a scalar transfer function for the system cannot be described 
and therefore the generic form has to be utilized to obtain the special 
forms . 

When components are interconnected it is possible to derive the 
differential equations for the combined system and then identify new 
variables and state equations. This involves a lot of unnecessary work 
and the original states lose their identity in the new system. Also, 
if the original component states are physically measureable quantities, 
this might not be true for the new states of the system. In the next 
section it is shown how the state equations for the system can be de~ 
rived from the component state equations without changing the original 
states. The generic form will be used for simple interconnections. 
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1.3 Procedure for Obtaining System State Equations . Consider the state 
equations for two components each designated by a superscript within a 
parentheses . 



(D = A (1) X (1) 
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Equations (1-16) through (1-19) can be combined into two matrix 
equations as follows: 
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Equations (1-20) and (1-21) imply that components one and two are 
operating independently, i.e., they are uncoupled. The next several 
sections investigate the effect of the following types of inter- 
connections . 
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(a) Cascade Combination 

(b) Parallel Combination 

(c) Feedback 

Figure 1-1 is a block diagram representation of Equations (1-1) and 
(1-2) for a component. The double lines indicate that the signals are 
matrix quantities . Equations (1-1) and (1-2) can also be expressed in 
flow graph form as shown in Figure (1-2). The flowgraph form is used 
in the following development to obtain the system state equations. 

When two components are cascaded, Figure 

( 1 - 22 ) 
(1-23) 
(1-24) 



1.3.1. Cascade Combination . 
1-3 results. From Figure 1-3 
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Substituting Equations (1-22) through (1-24) into Equations (1-16) through 
(1-19), one obtains the following new form for Equations (1-20) and 
d-21): 
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The identical result is obtained by inspection from the matrix flowgraph 

of Figure 1-4, i.e., Equations (1-25) and (1-26) are obtained by writing 

the matrix equations of the new dependent variables as functions of the 

4 

new independent variables. An interesting modification to Equation 
(1-20), due to the cascade connection, is that the lower left 0 matrix 
is replaced by a coupling term. Thus, the lower left and upper right 
0 matrix positions represent coupling of two components. Coupling 
connections are expressed in a general form in Figure 1-5. Note that 
the cascade combination is a special case of Figure 1-5. Cascade 
connection, when one of the two components is a matrix of gains, K, 
results in the following system state equations: 
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Parallel 


Combination. 


When 


two 


components are connected in 



parallel, Figure 1-7 is obtained. From this figure 
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Substituting Equations (1-29) and (1-30) into Equations (1-16) through 
(1-19) yields the following new form for Equations (1-20) and (1-21): 
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The same results are obtained by inspection of the flowgraph of Figure 
1-8. As one would expect from paralleling two components, the output 
is merely the sum of the outputs of the two components operating 
independently. 




1.3.3 State Variable Feedback . Introduction of state variable feed- 
back to a single component is shown in Figure 1-9. From this figure 

i 

0 ) ti) 

0 * 4-HX (1-33) 

where H is a lxn row vector of gains. Combining Equations (1-33), (1-16), 
and (1-17), one obtains the following results: 
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Using the flowgraph method and Figure 1-10, the same results are obtained 
by inspection. Introduction of state variable feedback changes the A and 
C matrices and thus enables one to move the open loop poles of the 
component to any desired location as shown by the system transfer 
function (for a single input and single output system). 



^ = cfSI-A]" 1 B + D (1-36) 
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where 



C = c (1 ^ - D (1 ^H 


(l-36a) 


A = - B^H 


(l-36b) 


B = B*' 1 '* 


(l-36c) 


D = 


(l-36d) 



Unity feedback is a special case of state variable feedback and there- 
fore treated the same. To demonstrate the use of the above, several 
examples are considered. Appendix A gives a set of state equations 
for a few networks and machines. Normally the states of a component 
are arbitrary; but, for the components listed in Appendix A the states 
chosen are physically measureable quantities. 



Example 1.1 Figure 1-11 is an example of a system formed by cascading 
several components. The state equations for the lead network (Appendix 
A) are 
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The state equations describing the servomotor are 
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(1-40) 
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Using Equations (1-25) and (1-26), the combined system state equations 
can be written directly. 
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V c (1-42) 

T is the gearbox ratio. The important result here is that system state 
equations are written directly without using the transfer function of 
the components and therefore retaining the original component states. 



Example 1.2 Unity feedback is now added to the system of Figure 1-11, 
and the resulting system is Figure 1-12. This type of feedback is the 
same as state variable feedback with a feedback gain matrix 



H = £ 0 T oj (1-43) 

Therefore, using Equations (1-34) and (1-35), the system state equations 
are 
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(1-45) 



The examples above illustrate the simplicity in obtaining the system 
state equations from the component state equations. The very same 
results are obtainable through the use of transfer functions but with 
the added difficulty of identifying the original states. The general 
equations used to obtain the system state equations do not have to be 
remembered because they are easily obtained from the flowgraph method. 

1.3.4. Complex Combination . More complex control systems often have 
component interconnections that are not entirely cascade, parallel, or 
feedback connections. The overall system may consist of components 
which are multi-input/multi-output, single input/single output, or 
combinations of both. Figure 1-13 is an example of a complex combination. 
The state equations for the overall system can be obtained by the flow- 
graph method. However, as the number of inputs and outputs of a com- 
ponent increases the flowgraph method becomes impractical. A method 
is proposed which will enable one to obtain the system state equations 
for any complex system as well as the cascade, parallel, and feedback 
connected systems. The general formulas for computing the system state 
equations are now derived. 

If the state equations of the individual components are combined 
into one large matrix of equations so that the overall system inputs 
are at the top of the input vector /£ and all other internal inputs 
form the bottom of the SZ vector, then the combined matrix of state 
equations can be written as 
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(1-46) 



X = AX + Ba /t a + Bb /J b 

is the vector of external system inputs and Sib is the vector of 
internal component inputs. The same arrangement of the input vector 
allows the combined matrix of output equations to be written as 

Y = C X + + D b ^6 (1-47) 

From the diagram of the system, the Sib vector can be written in terms 
of the Y vector of the combined system. This relation is 



- A Y (1-48) 

The elements of A , the interconnection matrix, are determined by 
inspection. To illustrate the procedure of writing the combined matrix 
of state equations, output equations, and the interconnection matrix, 
consider the system of Figure 1-13. For this example there are only 

For simplicity, consider each 
component to have only one state (first order components). Equations 
(1-46) through (1-48) become 
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If Equation (1-48) is solved for Y and this result equated to Equation 
(1-47), can be obtained in terms of x and A a. Substituting this 

result f or (i b into Equations (1-46) and (1-47) yields 



A+Bbd-ADbjAcjx + [b^ + B b (I - A Db) A Da. J /Z t 



I =[c+ Ob(l-ADb)'AC]x f [D*+ Dbd-ADbjADa. j/^ 



(1-52) 



(1-53) 



The complete derivation of Equations (1-52) and (1-53) is given in 
Appendix E. The above equations represent the state and output equations 
for any system. To illustrate the application of the above results, 
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consider the following examples. In each of these examples the elements 
of the A, B, C, and D matrices of the component state equations are 
arbitrarily chosen for illustration purposes only. 



Example 1.3 The combined matrix state equations and output equations 
for the system of Figure 1-14 are 
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The interconnection matrix equations are 
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Application of Equations (1-52) and (1-53) yields 
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Example 1,4 The combined state equations and output equations for the 
system of Figure 1-15 are 
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(1-65) 
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Again, application of Equations (1-52) and (1-53) yields 
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Example 1.5 . Figure 1-16 is different from the previous two examples 
because a feedback loop is present. From flowgraph theory one would 
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suspect that the determinant of (I - AD b ) can have a value different 
from unity. This is indeed the case. By inspection of the block 
diagram of the system for a closed loop, the necessity for computing 
the determinant of (I - ) can be established. Figure 1-16 has a 

closed loop provided is a function of fi and y^^ is a function 



of fl 



(3) 



The combined state equations for this system are 
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Substituting into Equations (1-52) and (1-53) yields 
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Example 1.6 This example demonstrates the procedure for obtaining 
the system state equations when the components are connected by pure 
gains and summers. Consider the system of Figure 1-17. The combined 
state and output equations are 
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For this system the interconnection matrix is 
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Again, application of Equations (1-52) and (1-53) yields the following 
state and output equations: 
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The examples above demonstrate the usefulness of the general method to 
a system composed of multi-input/multi-output components which are 
connected in any complicated fashion. For systems composed of many 
components, the method is particularly suitable to the digital computer. 

In this chapter a brief introduction to the state variable 
description of a dynamic system is presented, and some general methods 
are presented for obtaining the state equations for the system from 
the component state equations. This system description lends itself 
to a time domain design procedure. The classical approach to the design 
of a regulator or servomechanism uses the transfer function description. 
The question now arises as to how one designs a system in the time 
domain to meet required specifications? This is discussed in Chapter II. 
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Figure 1-2. Flowgraph of State Equations 
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Figure 1-3. Cascade Connection of Two Components 
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Figure 1-4. Flowgraph for Cascade Combination 
(I is the Identity Matrix) 
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Figure 1-5. Block Diagram of Coupling 
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Figure 1-6. Flowgraph for a Cascaded 
Pure Gain Component 
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Figure 1-7. Block Diagram of a Parallel Combination 
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Figure 1-8. Flowgraph for a Parallel Combination 




Figure 1-9. Block Diagram for State Feedback 
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Figure 1-10. Flowgraph for State Feedback 




GecL/i I 

SoX PloaJ 



Figure 1-11. Block Diagram of Cascaded System 



35 




1 Li:i r* l-i: . "lo 1 r: *im ol Unity V c^!nn:k 




;■ igure 1-13. Complex Combination 



36 




Figure 1-14. Example 1.3 




Figure 1-15. Example 1.4 
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CHAPTER II 

System Design Using Optimization Theory 

2*1. Introduction . In Chapter I the state space description was 

used to represent the control system. A design technique or approach 

which is based on this type representation is known as optimal control 

theory. Even though some of the optimal criteria are stated in the 

3 

frequency domain, most are in the time domain. Kalman argues that 
this is a weakness of the modern approach because the majority of the 
control concepts are expressed more simply in the frequency domain. 

The experienced designer of a servomechanism or regulator uses fre- 
quency domain concepts such as root locus, phase margin, bandwidth, 
etc., because he can relate these parameters to a good design. To 
use the modern approach, the designer must translate the design spec- 
ifications into a performance index which he then proceeds to minimize 
or maximize to obtain the control that yields these specifications. 

The correlation between the performance index and performance criteria 
is the main subject of this chapter. In order to study the problem, 
three subobjectives are considered. 

(1) To obtain the optimal control in terms of the system and 
performance index parameters for a simple system. The motivation 
behind this objective is the desire for a better understanding 

of the effect of these parameters on the resulting optimal system. 
This type of information is helpful in translating specifications 
into a performance index. 

(2) To study the evaluated cost function in order to obtain its 
meaning in terms of a classical control problem. 



39 



(3) To investigate the optimal design techniques that are used 
to realize a desired system. 

The infinite linear regulator problem is considered because it has a 
closedform mathematical solution and it is similar to the servomechanism 
prob lem. 

2.2. Derivation of the Optimal Control . The purpose of this Bection 
is to derive the optimal control for the linear regulator of Figure 2-1 
with the following performance index: 

J = J [X T QX + tlVliUd-fc. (2-1) 

0 

Q is a positive semidefinite state weighting matrix and P is a positive 
definite control weighting matrix. The integrand of Equation (2^1) is 
positive definite to ensure stability of the optimal system. The plant 
to be optimized is described by 

*: = AX+ Bij (2-2) 

The above performance measure expresses the desire to drive the state 
vector from an initial condition to the origin with a trade off between 
the system error and the amount of control energy expended. This desire 
is readily seen by making use of the end result. Minimization of this 
performance index leads to time invariant linear state variable feed- 
back as the optimal control. For a second order linear time invariant 
plant the optimal control is 

U = -K,X, - K Z X, (2-3) 
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Substituting this result into Equation (2-1) and expanding yields 



+■ Jp,,[Kl'Xr+2K l K l X ( X z f KiXtJdt 



(2-4) 



where 



Q 



q ll ° 

0 q 22 



(2-5) 



P = 



P 11 



(2-6) 



The first integral of Equation (2-4) represents the system error and 
the second integral represents control energy. The choice of q^, ^ 22 ’ 
and p^ represents the emphasis on minimizing the system error as 
opposed to control energy. 

The control interval is infinite and thus time is of no importance. 
For a finite upper limit on the performance index, the optimal control 
is time varying state feedback. In the optimal control literature it 
is shown that the optimal control is 



UC&J o = -pVr «)& 



(2-7) 



where R(t) is the solution to the matrix Ricatti equation. 



Q - FUOBpVrk) + RU)A -t- ARU) = O (2-8) 



Since the performance index has an infinite upper limit and the optimal 
system is assumed to be stable, R(t) vanishes and R(t) becomes a time 
invariant matrix. This leads to the reduced Ricatti equation. 



A T R +-RA -RBPB T R + Q - Q 



(2-9) 
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The optimal control. Equation (2-7), then reduces to 



* - p'b r X 



( 2 - 10 ) 



For the plant of Figure 2-1, the matrices of the reduced Ricatti 
equation are 
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( 2 - 12 ) 
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(2-15) 



R is a symmetric positive definite matrix. Substituting the above 
matrices into the reduced Ricatti equation yields 




+ #0 


(l// - CL/lru j n /In /In) 




0 


O 


•£*■**“) 


fy-Q.iir'kuhc 




O 


o 

J 



(2-16) 



42 



Application of Sylvester* s Theorem to obtain the requirements for a 
matrix to be positive definite yields the following solution to the 
Ricatti equation: 
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(2-17) 
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The optimal control is 
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This result leads to the system in Figure 2-2 where 

K, = Jfrfc 



( 2 - 21 ) 



( 2 - 22 ) 



(2-23) 
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(2-24) 
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Equations (2-23) and (2-24) represent optimal feedback gains in terms 
of the plant parameters and the weighting factors of the performance 
index; but before discussing them, another approach is employed to 
check the results and show the effect of the parameters on the state 
trajectories . 

A check on these results is obtained by using the Euler equation. 

However, this approach is valid only if the plant can be described in 

Normal (phase variable) Form which implies that the plant must be 
3 

controllable. For the plant in Figure 2-1 

X, - X* (2-25) 



X* - - AXi 



(2-26) 



or 



Xi ” 4“ C* M 



(2-27) 



The performance index, Equation (2-1), can be written as 

J= j CgiiXf 

0 



(2-28) 



Solving Equation (2-26) for u and substituting the result into Equation 
(2-24) yields 
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** + 
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fu X,X t * A* 1 dt 



(2-29) 
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or 



oO 

J = Ig ^D^.»X,)clt (2-30) 

o 

Substituting the integrand above into the following Euler equation 



a X, 



A 






&9(Xi.x.,x,) 


dt 


L ^X, _ 


+ d-fc* 


c) X, 



= O 



(2-31) 



yie Ids 



X, -[ <5 '^+ a -]x, -I- X, - o 



(2-32) 



Taking the Laplace transform of Equation (2-32) yields 
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(2-33) 



Equation (2-33) can be factored into the following: 






1 U-JA- B 'l|s+A Wa 1 - B =P 1 Is-'/aWa x - 



= O (2-34) 



where 

A = + a (2-35) 

& s 6 Z ^i. (2-36) 

P" 
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Two of the roots of Equation (2-34) are in the right half plane and 
thus must be eliminated to satisfy the boundary condition 



X M - Q 



(2-37) 



Thus, Equation (2-34) reduces to 

[s-Ja-Ja'-B 1 '][s+ /aW/^B^] =o 



(2-38) 



Taking the inverse Laplace transform yields 



X, f X, 
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-f x,/eT = o 



(2-39) 



Equation (2-39) represents the optimal state trajectory for the system 
based on the chosen performance index. But, from Equation (2-27) 



X, - - & X ( + £ M 



(2-40) 



Substituting this equation into Equation (2-39) and solving for u 
yie Ids 

Therefore, the optimal feedback gains are 







(2-42) 
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(2-43) 
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Equation (2-43) is simplified as follows: 



(K 2 & + a) z Jza+z/F 



(2-43a) 



Solving for yields 




r-a+jlA+I/F 



(2-44) 



With the appropriate substitutions for A and B, the optimal feedback 
gains are 



These results are identical to those found by solving the reduced 
Ricatti equation. 

The following results are stated without proof for a first order 
plant shown in Figure 2-3: 



The above approach not only serves as a check but also demonstrates the 
effect of the plant and cost index parameters on the optimal trajectories. 
To obtain general results such as these for higher order systems brings 
about mathematical difficulties. The digital computer must be employed 
to obtain specific results for specific systems. The solution to the 




(2-45) 




(2-46) 




(2-47) 
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reduced Ricatti equation is obtained by numerically integrating the 
matrix Ricatti differential equation backwards in time with the 
following boundary condition: 

R (t = 0) = 0 (2-48) 

For large negative time the R terms will approach constant values that 
are the solution to the reduced Ricatti equation. An example of this 
procedure is given in section 2.3. However, the general results obtained 
for the second order system serve to illustrate the effect of the plant 
and performance measure parameters on the Optimal system. 

With attention focused on Equations (2-45) and (2-46), some 
general results are obtained. If the elements of the Q matrix are 
held constant and p^ made large, the feedback gains are small. This 
result is reasonable since large weight on control energy relative to 
the weights on the system error expresses the desire to use small 
amounts of control energy at the expense of large states, i.e., system 
error. For a constant p^ and for large values for the elements of the 
Q matrix, the feedback gains are large. Again this result is reason- 
able since the emphasis is now shifted to small system error at the 
expense of large controls, i.e., large feedback gains. However, the 
Q and P matrices actually have the same effect on the feedback gains 
since the optimal control is a linear combination of the states. That 
is, gains are made large by increasing Q or decreasing p^. The same 
optimal system results from an infinite number of choices of Q and p 
since the optimal control is a function only of the ratios ^l/p^ 
and ^ 22 ^ 11 * ^us > unless each state is of separate concern, the Q 
matrix should be the identity matrix and the p^ term used to achieve 



48 



the desired optimal system. This result will be utilized in the section 
on modeling. The pole and gain of the plant also affect the feedback 
gains. For large pole values the size of K 2 decreases. This is expected 
because large pole values imply a small time constant for the open loop 
plant. Thus, the plant is able to decay to the origin with little 
assistance from the control. 

The characteristic equation describing the system in Figure 2-2 
is 

K, + (a+ K*6)X, f K,6 X, -O (2-49) 



Therefore 



- 






a+ Kz G _ 




(2-50) 



(2-51) 



where Wn is the undamped natural frequency and ^ is the damping ratio. 
From Equations (2-46) and (2-47), large Q matrices relative to p^ 
produce highly damped systems with large bandwidths. But note that 
can exceed unity. One would expect that some sort of limit on ^ 
would result from the optimization process. For large values of p^ 
relative to the terms of the Q matrix, optimal systems which are highly 
oscillatory with small bandwidths areproduced. But, there is a com- 
promise between these two extremes which is obtained by trade-offs 
between q^, q^ 2 , and p^. Thus, even though the optimal control has 
been found, the classical ideas on what constitutes a good design 
must be introduced to express the ultimate objective. The performance 
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index chosen yields a class of optimal systems and any specific one of 
the class is chosen by selecting the appropriate Q and V matrices. 
However, the general results obtained for the second order system do 
give some feeling on how to select the parameters of the performance 
index. For higher order systems the exact relationships between the 
performance index parameters and the classical parameters are not known, 
and thus some other approach must be taken. This new approach is 
modeling. 

2.3 Mode llnK * As noted in the previous section it is difficult to 
translate the classical design criteria into the performance index 
parameters for systems higher than second order, Even the procedure 
of adjusting the Q and P matrices until an acceptable time constant 
and percent overshoot are obtained gives no information about the band- 
width, phase margin, etc. However, this problem can be eliminated by 
using a procedure similar to one employed by the classical approach. 

In the classical approach, one often adds components or feedback loops 
so that the new system behaves like a known differential equation. 

This model differential equation represents a system that has the desired 
classical specifications. If this model is incorporated into the per- 
formance index so that the mathematical process of minimisation is 
applied to the difference between the system response and the model 

response, then the optimization theory takes on meaning and practical 

14 

significance. Schultz and Melsa present such a modeling approach 
to the design of a linear regulator, 

Consider a plant described in Normal Form. Any quadratic index 
of the form 




(2-52) 



O 



50 



where the states are phase variables, is reducible to 



oO 

T = J"l- £ L X) + (x $<X) +- /U Z J dt. (2-53) 

o 

S is a symmetric constant matrix of order one less than the system, and 
L is a column vector of system order. That is, the first term of the 
quadratic form is reducible to a perfect square and an exact differential. 
Again, it must be emphasized that this result is valid only when the 
states are phase variables. The exact differential is not affected by 
the minimization process because 




t 







t-o 






(2-54) 



From Equation (2-54) it is evident that the exact differential term 
represents a fixed value of cost that is determined only by the initial 
conditions. The term is positive because the terms of the S matrix 
are always negative. The derivation of the L and S matrices for second 
and third order systems is given in Appendix B. The general recursion 
formula for higher order systems may be obtained from Reference 14. 

The performance index, for the Normal Form representation, is equivalent 
to 

tfO 

J= L T X) + p> t W*] dt (2-55) 

c 

With this performance index, the L matrix has to be selected instead of 
a Q matrix. But this form allows one to incorporate the desired model. 
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Consider, for the moment, that p is zero. Then the cost function is 
zero if 

Lx= flX.U) + ^ X, It) + + h X, Lt) - o (2-56) 

for all time. By the proper choice of the L matrix, Equation (2-56) will 
be the model differential equation. The Q matrix, which is needed to 
solve the Ricatti equation, is derived from the L matrix using the 
formulas of Appendix B. Then, p^ is decreased until the optimal system 
response is close to the model response. However, the system response 
cannot be identical to the model response because as p^ approaches 
zero the feedback gains become extremely large. Stability problems are 
also encountered due to the fact that p^ is then semidef inite • The 
value of the cost function then represents how close, on an integral 
square error basis, the system response is to the model. Thus, one is 
prevented from attaining the desired model response by physical restric- 
tions on the size of the feedback gains. Note that the model equation 
must be at least one order less than the plant in order that the L 
matrix not exceed the system order. This presents no problem since one 
usually designs higher order systems to behave like second order systems. 
This modeling technique is also applicable to plants which have zeros. 

In summary, the procedure of this technique is to select the L matrix 
from the coefficients of the differential equation representing the 
desired response and then determining the elements of the Q matrix from 
the relations in Appendix B. With this Q matrix, the Ricatti equation 
is solved with various values of p^ until the maximum permissable 
feedback gains are obtained. The following examples demonstrate this 
procedure . 
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Example 2,1 



Consider the plant of Figure 2-1 where 



G = 1 ; a = 2 (2-57) 

and the desired model is 

X L + 2X l = 0 (2-58) 

Since the L matrix comes from the model differential equation, it 
follows from Equations (2-56) and (2-58) that 



L = 



2 

1 



(2-59) 



Using this L matrix and the relations of Appendix B, the Q matrix is 
determined to be 



Q = 



4 

0 



0 

1 



(2-60) 



With this Q matrix, the feedback gains are determined from the relations 
for the second order system of the previous section. Figure 2-4 shows 
the desired model response as well as the optimal system responses for 
various values of P-q* Note that as p^ is decreased the optimal system 
response approaches the model response. Figure 2-14 shows the trade- 
off between the size of the optimal gains and the cost (proximity of op- 
timal response to model response). The next example is a more practical 
one since the plant is third order and the desired model is second order. 



Example 2.2 Consider the plant of Figure 2-5 where 

G=l;a=3;b=5 (2-61) 



and the desired model is 

+ 2X l + 2X 1 = 0 



(2-62) 
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Using Equations (2-56) and (2-62), the L matrix is 



L = 



2 

2 

1 



(2-63) 



From the results of Appendix B for a third order plant, the Q matrix is 
determined to be 



Q = 



4 0 0 
0 0 0 
0 0 1 



(2-64) 



Since there are no closed form solutions to the reduced Ricatti equation, 
the matrix Ricatti differential equation must be numerically integrated 
to obtain the solution to the reduced Ricatti equation. This solution 
is obtained by integrating the matrix Ricatti differential equation 
backward in time with initial conditions 



R (t = 0) = 0 (2-65) 

For sufficiently large negative time, the terms of the R matrix will 
approach constant values and these values are the solution to the re- 
duced Ricatti equation. The optimal feedback gains are then obtained 
from 

K = R B P -1 (2-66) 

The solutions to the matrix Ricatti differential equation for various 
values of p ^ are shown in Figures (2-7), (2-8), (2-9), and (2-10). 

The system responses for the feedback gains associated with the various 
p^ are shown in Figure (2-11). Again, note that as p^ decreases the 
optimal system response approaches the model response. Figure 2-15 
shows the trade-off between the size of the optimal gains and the cost 
(closeness of the optimal response to the model response). The 
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mathematical formulation of the first order differential equations com- 
prising the matrix Ricatti differential equation is given in Appendix C. 
This modeling technique provides more information about the meaning of 
optimality with respect to a quadratic index than the techniques to be 
discussed next. Since the Q matrix and the p term affect the optimal 
control in the same way, it is simpler to fix the Q matrix and vary 
the p^ term to obtain the desired optimal system. Once the Q matrix 
is chosen, optimality is also defined since there is a unique L matrix 
associated with the chosen Q. 

9 

Another modeling technique is proposed by Tyler . This technique 
also uses the quadratic performance index but defines a new set of state 
variables that are the difference between the model states and the 
states of the plant. This index is then minimized. This procedure also 
leads to the matrix Ricatti differential equation. This procedure, 
unlike the previous one, requires the adjustment of both the Q and P 
matrices to yield a satisfactory system response. These two modeling 
techniques bring to light several conclusions about the optimal control 
approach to the design of regulators and servomechanisms. The optimi- 
zation technique is a mathematical method of introducing a set of 
variable parameters into the actual system that provide the designer 
with a more useful tool with which to cope with system constraints such 
as gains. However, the ultimate objective is still expressed in the 
form of a differential equation. The optimal controls approach is a 
search for the performance index that yields the desired differential 
equation. The physical significance of the value of the cost index is 
also questionable and, therefore, is considered next. 
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2.4 The Evaluated Cost Index. 



Consider again the performance index 



of the linear regulator. Kalman shows that the value of this index 
for a stable system is 

J = X T R X (2-67) 

op - 

where R is the solution to the reduced Ricatti equation. For the second 
order optimal system of Figure 2-2, Equation (2-67) yields 

Jip = p„ K- (a 4 K 2 6)X?(o) + Hi X,Lo)X z (o) -t- p.. H X^o) (2-68) 

Section 2.1 stressed the fact that there are many values of Q and P 
that yield the same optimal system. This is also true for the value 
of the cost index. One could then argue that the value of the cost 
index has no significance. In general this is true. But, for the 
modeling technique of Schultz and Melsa, the cost does have meaning. 

For that technique, zero cost corresponded to a system response identical 
to the model response. Thus, the cost represents the deviation of the 
system response from the model response in an integral square error 
sense . 

Even though the numerical value of the cost provides little infor- 
mation of interest in the design problem, it does provide some insight 
into the mathematical process of minimization. The cost function 

c?0 

T= X T QX -H ^U T p^U)c)t (2-69) 

is evaluated for a non-optimal second order system by assuming solutions 
for X^(t), X 2 (t) , and as follows: 

At -^t 

X, Li.) = <X,e + ' (2-70) 
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( 2 - 71 ) 



C i£) - - “ <£ ,A(& - ^2^2. 

Wltv- -K,X,iO- K^X^O (2-72) 

The feedback gains of Equation (2-72) are not the optimal gains and thus 
are not the same as Equations (2-45) and (2-46)* For the moment, 
consider them as arbitrary values. Substituting these solutions into 
Equation (2-69) and carrying out the integration yields 



T- c K*+ gu ) + OC+ Cl+ \< z 6 ct + + fu.) 

P" ' r P" / 



K,£t 



ZK lKt( at Ki 6 i]|^^ Ng+|<) 



(2-73) 



4 - 



K ' Z+ % + Kz + p* 



ft. Xx(c) 

p'»- zca+kt.6 



L \<^c^ 

Equation (2-68) may be obtained from this relation by substituting the 
optimal gains. However, note that Jop, Equation (2-68), is a positive 
definite function, whereas J, Equation (2-73), may not be positive 
definite. The positive definite function, Jop, plots on the versus 
plane as a family of concentric ellipses as shown in Figure 2-12. 
The constant cost curves decrease continually toward the origin. How- 
ever, Equation (2-73) will plot as a family of curves that may overlap 
since it is not necessarily a positive definite function. If the 
optimal feedback gains are used, the state trajectory on the X^ versus 
X^ plane will cross the family of constant cost curves in a manner 
such that the cost at any instant of time on the trajectory is less 
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than that for any previous instant of time. But, if non-optimal gains 

are used, this will not be true. The reason it is true for the positive 

3 

definite function is due to a theorem by Kalman which is discussed in 
Chapter III. The theorem states, under certain assumptions, that Jop 
is a Lyapunov function and that 

df- Jop = - [ X T QX + y T P,y] (2-74) 

is a negative definite function because of the previous assumption on 
P and Q. Therefore, the state trajectory must transit the Jop curves 
in the manner described above. One might suspect that the state tra- 
jectory on the versus plane would be normal to the Jop curves. 

This is not true because 



dt 



^ Jop 




I 


1 — 

Ox 1 

ix ! 


dt 


L h'L. 



X 



(2-75) 



Thus, as seen by Equation (2-75), the time rate of crossing is not only 
a function of the gradient of the Jop curves but is also a function of 
the state equations. The shape and amount of axis rotation of these 
Jop curves depends on the choice of Q and P, as well as the plant para- 
meters. Thus, when more weight is put on one of the states than the 
other, the effect is a rotation of the axes of the family of cost curves 
so as to restrict the motion of the state trajectory. Figure 2-13 is 
a plot of these cost curves along with the optimal trajectory for 
Example 2.1. The value of p^ for Figure 2-13 is unity. 

A reasonable conclusion to be drawn from this chapter is that in 
the design of a regulator or servomechanism the optimal controls approach 
is superior to the classical approach in some respects and inferior in 
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other respects. The optimal controls approach does allow the engineer, 

inexperienced in the classical techniques, to design a control system 

to meet the required specifications. With the use of the modeling 

14 

technique of Schultz and Me Isa , the modern approach gives the engineer 
a single number, the cost, that represents the proximity of the optimal 
system response to the model response (integral square error sense). 

The implementation of the optimal control is another problem. The 
finite linear regulator problem leads to an optimal control that is 
time varying state feedback. The servomechanism problem has an optimal 
control which is time varying state feedback plus a function of the 
desired reference states. The synthesis of these controls is difficult. 
Therefore, in most cases, the experience factor is merely shifted to 
the synthesis of the optimal control. The linear regulator was con- 
sidered in this chapter because of the simple mathematics involved, 
and also because it offered a challenge to the meaning of optimality 
with respect to the design of a regulator to meet certain specifications. 
The quadratic performance index was discussed in detail. The effect of 
block diagram manipulations on this index is considered in Appendix D. 

This chapter used the linear regulator to study the meaning of 
optimality. For this particular problem, the quadratic performance 
index with a finite upper limit led to a time varying system. When 
the optimal controls approach is used, the resulting optimal system is 
usually non-linear, time varying, or both. Stability then becomes a 
subject of primary concern. Lyapunov stability theory is the primary 
tool that is used to study the stability of these type systems. Like 
the optimal controls approach, Lyapunov stability theory is based on 
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the time domain description of systems. Chapter III will introduce 
some of the stability criteria that are useful in determining the 
stability of a class of time varying systems for which stability can 
be studied directly. 
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Figure 2-1. Second Order Plant 




Figure 2-2. Optimal System 







Figure 2-2. First Order Plant 
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Figure 2-4. Optimal System Responses 
and Model Response 
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Figure 2-5 . Third Order Plant 




Figure 2-6. Optimal System 
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Figure 2-7. Steady State Solution to Ricatti 
Equation for = 1.0 
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Figure 2-8. Steady State Solution to 

Ricatti Equation for - 0.1 
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Figure 2-9. Steady State Solution to 

Ricatti Equation for p = 0.01 
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Figure 2-10. Steady State Solution to 

Ricatti Equation for = 0.001 
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Figure 2-11. Optimal System Responses 
and Model Responses 
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Figure 2-12. Curves of Constant Cost (C 2 >C 2 >C^) 
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Figure 2-13. Optimal vs Trajectory 
with Constant Cost Curves 
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Figure 2-14. Cost Versus Optimal Feedback 
Gains for Example 2.1 
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Figure 2-15. Cost Versus Optimal Feedback 
Gains for Example 2.2 
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CHAPTER III 



Stability of a Class of Time Varying and Optimal Systems 

3.1. Lyapunov's Second Method . In the analysis and design of 
control systems an important question to be answered in the early stages 
is the question of stability. Not only is a yes or no answer desired, 
but also the limits of stability must be investigated. Various design 
criteria such as phase margin and the Routh criterion express the limits 
of stability graphically and algebraically, respectively. Thus, the 
range of values that system parameters can take on is of vital impor- 
tance in the design problem. The study of stability can be broken 
down into two general areas. First, the stability of linear systems 
is straightforward due to the Routh and Nyquist criteria. Second, 
the stability of non-linear as well as time varying systems is not 
straightforward and requires the use of Lyapunov's methods because the 
Routh and Nyquist criteria are no longer applicable to these type 
systems. The first method or indirect method of Lyapunov is so named 
because it requires the solution to the non-linear differential 
equation. This method investigates the stability in a small region 
about each of the equilibrium states of the system. However, knowledge 
of this type stability is usually not enough in certain problems of 
interest. The second method or direct method is a tool which enables 
one to obtain more stability information than is obtainable by use of 
the first method. The second method of A. M. Lyapunov, published in 
Russian in 1892, was developed for the purpose of studying the stability 
of mechanical systems. It was not applied to electrical systems until 
1944, The second method is applicable to linear (time invariant and 
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time varying) and non-linear systems. For linear, time invariant 
systems, it can be shown that the Routh criterion and the second 
method impose the same requirements for stability. 

As mentioned, the second method is not as straightforward as the 
Nyquist or Routh criteria. Stability, by this second method, is 
assured provided there exists a so called Lyapunov function which be- 
haves in a prescribed manner. The determination of the stability of 
a system consists of a search for a non-unique Lyapunov function, 

V(x, t), which is a function of the system states or a function of 
the states and time. The Lyapunov function is often compared to the 
energy in a system. However, the energy in a system is not necessarily 
a Lyapunov function because it does not have to be a decreasing func- 
tion of time as a Lyapunov function. That is, the average energy of 
a system may be a decreasing function of time, but the instantaneous 
energy may not be. Such a system would be unstable and thus the energy 
would not be a valid Lyapunov function. Since Lyapunov functions are 
not unique, the determination of the limits of stability is not an 
easy task. One may obtain a valid function that establishes a set of 
limits for stability, but nothing can be said about the stability be- 
yond these limits. That is, the function assures stability within the 
limits but does not assure instability beyond. In the literature there 
are two basic methods proposed for generating Lyapunov functions, the 
gradient method and the Lur*e method. As might be suspected, since 
both meithods are straightforward, their requirements on the system are 
too restrictive. Because of this, and the fact that all non-linear 
systems cannot be treated in the same way, non-linear systems are 
broken down into classes depending upon the type of non-linearity. 
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This enables one to establish a stability criteria that is not so 
restrictive. In this chapter stability criteria that have been de- 
veloped for linear systems with a time varying gain in the single 
feedback loop are considered. The stability of optimal systems is 
also discussed. Before doing this, it is necessary to state the types 
of stability which will be of concern. 

3.2 Stability Definitions . Before stating the applicable stability 
definitions, the terminology used in stability analysis of non-linear 
and time varying systems is reviewed. A non-linear or time varying 
system is described by the following state equations: 

X = f(X,t) (3-1) 

where f is a non-linear function of the states and time. The solution 
of Equation (3-1) is writen as 

(f)(t; X Q , t Q ) (3-2) 

where X is the state vector at time t or 
— o o 

(t)(t ; X , t ) = X (3-3) 

o—o o —o 

The equilibrium states, _Xe , of Equation (3-1) are all values of _Xe 
such that 

fOLe, t) = 0 (3-4) 

Non-linear systems may have one or more equilibrium states and each 
may be shifted to the origin (Xe = 0) by an appropriate change in 
state coordinates. The norm of a point or vector in the state space 
is a function which assigns to that point a real number such that 

( 1) || X|| > 0 for all x 
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( 2 ) 



for all x, 



II* + ill ^ ||x|l + Hill 

o) iuxil= U. Mix II for all x and constants oC 

(4) || X || = 0 implies x = o 

A motion is defined as the trajectory starting at any point in the 
state space. There are many types of stability; a system may be stable 
in the whole state space or stable only in certain regions. The system 
may be stable in the sense that the state vector decays to the origin, 
or in the sense that the state vector is bounded by a finite region. 

The many variations of stability are discussed by Kalman and Bertram^ 
and by Hahn^. For purposes of this chapter the following definitions 
are applicable and are taken from Reference 17. 



Definition One . An equilibrium state xe of a free dynamic system 
is stable (in the sense of Lyapunov) if for every£>0 there exists a 
real number such that J| X q - xj| £> implies 



||(f)(t; X o , t o ) - X e ||< £ for all t> t o . 

If S is not a function of t^ the system is uniformly stable. Thus, 
for a system which is stable in the sense of Lyapunov, the state vector 
is confined to a finite region. 



Definition Two * An equilibrium state Xe of a free dynamic system 
is asymptotically stable if 

(1) It is stable and 

(2) Every motion starting sufficiently near Xe converges to 

Xe as t-> . Asympotic stability is of more practical importance 
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than stability in the sense of Lyapunov. However, an asympotically 
stable system may be an impractical design since the system may operate 
outside of the asymptotically stable region. The ultimate stability 
desired is asymptotic stability in the large which is asymptotic 
stability in the whole of state space. The main stability theorem of 
Lyapunov's second method is stated in Reference 17. This theorem 
establishes the requirements on the Lyapunov function for uniform 
asymptotic stability in the large. This is the most stringent of all 
the types of stability and, as a result, the requirements for the 
weaker forms of stability are obtained by weakening the conditions of 
the main theorem. With these few remarks about Lyapunov's second method, 
the various stability criteria for linear systems with a time varying 
gain in the feedback loop are investigated. 

3.3 Brockett and Fory's Stability Criterion . The system shown in 
Figure 3-1 has received a great deal of attention because of its 
appearance in control systems and in electronic devices. It might re- 
present a control system, a parametric amplifier, or other electronic 
device. Many authors have attempted to develop a stability criterion 
for this type system which establishes stability limits beyond those 
of the presently known criteria. Brockett and Forys^ have established 
a criterion for stability in the sense of Lyapunov (Definition One) 
for the system of Figure 3-1. Since the system is linear, although 
time varying, stability in the sense of Lyapunov implies boundedness 
for any initial conditions. Their criterion is based on the following 
theorem which is obtained from the main theorem of Lyapunov's second 
method . 
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Theorem One . The time varying system in Figure 3-1 is stable in the 



sense of Lyapunov if there exists a continuous positive definite 
function V(X) , having continuous first partial derivatives with respect 
to X, such that the time derivative of V(X) is non-positive along any 
solution of the system. 

Before stating the Brockett and Fory criterion, a few remarks 
concerning the development are in order. The system of Figure 3-1 is 
expressed in Normal or phase variable form such that the output is 
written as 

q(D) X = Y (3-5) 

where D is the familiar differential operator. The differential equation 
describing the system becomes 

p(D) X + f (t) q (D) X * 0 (3-6) 

Brockett asserts that if all solutions of Equation (3-6) are bounded, 
then the system of Figure 3-1 is also bounded. The stability criterion 
for the system is obtained from a study of Equation (3-6), The develop- 
ment is also based on the analogy between the systems of Figures 3-1 
and 3-2. That is, the system in Figure 3-1 is equivalent to a passive 
network with a time varying resistor across one of the ports. Based 
on this analogy, it seems reasonable to suspect that Equation (3-6) is 
stable if q(D)/p(D) is positive real and f(t) is non-negative. This 
analogy led Brockett to the following theorem: 

Theorem Two . The system in Figure 3-1 is stable in the sense of 
Lyapunov provided G(s)+ — is a positive real function whose real 
part is not identically zero and 0 £ f(t) £ K. 
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Theorem Two is a sufficient, but not necessary, condition. This 

theorem, although useful, severely restricts the linear part of the 

system. The Nyquist criterion, when applied to the same system with 

a time invariant gain, predicts stability provided the Nyquist plot 

does not intersect the negative real axis to the left of - — . How- 
ls. 

ever, the above theorem predicts stability if the Nyquist plot avoids 

the entire plane to the left of - ^ . Because of these severe 

is. 

restrictions, Brockett attempted to lessen them by placing restric- 
tions on f(t). The attempt led to the following theorem: 

Theorem Three . The system in Figure 3-1 is stable in the sense of 
Lyapunov if there exists an oC , ^ , and K such that 



(1) 


[G( s) 


1 1 /Tfl 1 + ^ S 


is positive real 


(3-7) 


+ 1/k j L 1 + p S 


(2) 


0 


^ f(t) < K 




(3-8) 


(3) 


kn 

f(t> 


- 2 [ 1 - f (t) /K ] 


min (-^ ) 


(3-9) 



This theorem provides sufficient conditions but not necessary conditions. 
This is to be expected since Lyapunov functions are not unique. To 
obtain the meaning of the above theorem it is necessary to intepret 
the equations and the constants oC and (3 . For the moment, consider 
oC and as an arbitrary constant and zero respectively. The 

positive real requirement of Equation (3-7) is easily established by 
introduction of the modified polar plot of Popov. This plot, shown 
in Figure 3-3, is the same as the polar plot except the ordinate is 
wlmG(jw) instead of ImG(jw). Equation (3-7) is positive real provided 
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(3-10) 



or 



RuG (jw) + 7 - oC uJ ImG( jw) — 0 

K 



(3-11) 



However, the equality portion of Equation (3-11) 



Re G( jw) + ~ - oC WlmG(jw) = 0 

K 



(3-12) 



plots as a straight line with a real axis intercept of — and slope 



^ on the modified polar plot. Therefore, the inequality of Equation 
(3-11) will hold only if the modified plot of G(jw) lies to the right 
of this line. Note that the above requirements only satisfy the non- 
negative real part restriction for a positive real function. Similarly, 
ior a zero oC and a non-zero ^ , Equation (3-7) is positive real 

provided the modified polar plot of G(jw) lies to the right of the 
line 



This line is also plotted in Figure 3-4. To apply the theorem, either 



would then reduce to Theorem Two. The selection depends on the modi- 
fied polar plot of G(jw). If the modified polar plot of G(jw) crosses 
the negative real axis with positive slope, [3 is selected to be zero. 
If the plot crosses with negative slope, oC is selected to be zero. 

This theorem allows one to reduce the restrictions on f(t) by placing 
restrictions ot f(t). This is the purpose of cC and ^ . That is, 

* for a system that has a zero ^ , the maximum restriction on f(t) 

* 

occurs for a zero <?C . Equation (3-9) then implies that f(t) is not 
bounded because is not hounded. However, as oC is increased 




(3-13) 




is selected to be zero. Both could be zero but the criterion 
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Equation (3-7) will be positive real for larger values of K and thus 
f(t). But f(t) must then take on smaller values as seen from Equation 
(3-9) . There is also an upper limit that f(t) cannot exceed and this 
is established by a line tangent to the modified polar plot at the 
negative real axis intercept. The slope of this tangent line puts 
an upper limit on dC and thus establishes the most restrictive set of 
bounds for f(t). Equation (3-9) is used to establish the most restric- 
tive set of bounds for f(t) in the following manner. After the maximum 
value of cC is determined as explained above, the upper bound on 
f(t) is determined from Equation (3-9) by substituting the maximum 
oC and the maximum K into the equation. The maximum K will be the 
reciprocal of the negative real axis intercept of the modified polar 
plot of G(jw). Equation (3-9), for these values of cC and K reduces 
to 



f(t) ^ 



2 f(t) 



r i i 


i 


1 K 

max 


max 



(3-14) 



6 

Up to this point the maximum restrictions on f(t) and f(t) have been 
established. A compromise between these two extremes is possible for 
the proper choice of <£ and K. This is the significant contribution 
of the theorem. The following example illustrates the explanation 
above . 



Example 3.1 Consider the system in Figure 3-1 where 

1 

G(s) = 2 (3-15) 

v ' s + ps ' 

The differential equation describing the system is 



X + pX + f(t) X = 0 



(3-16) 
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The modified polar plot of G(jw), shown in Figure 3-5, is a straight 
line passing through the origin with slope p . The problem is to 
determine the range of values of f(t) and f(t) for which the system is 
stable. Since the plot crosses the real axi6 with positive slope, ^ 
is taken to be zero and the equations of Theorem Three reduce to 

[g(s) 4 1/k] (1 4 s) (3-17) 

0 < f(t) C K (3-18) 

loo & ~k [ l ' f(t)/K ] <3 ' ,9) 

To obtain the maximum restriction of f(t) an equal to zero is always 
chosen. For this choice of ck , Equation (3-16) is positive real if 

K £ p 2 (3-20) 

Since the maximum value that K can assume in order that 

G(s) 4 1/K (3-21) 

is positive real is the minimum of the real part of G(jw), Equation 

(3-20) can be obtained from the modified polar plot. To obtain the 

maximum value of K , a vertical line tangent to the modified polar plot 

is constructed. The intersection of this line with the negative real 

axis yields — - — - . Note that this line has slope 
K 

max 

-L = oo 

A (3-22) 

and the modified polar plot of G(jw) lies to the right of it. Thus, 
this line represents the choice of oC and K such that i(t) has the 
maximum restrictions and f(t) has none. Equations (3-18) and (3-19) 
reduce to 

0 < f(t) C p ? (3-23) 
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f(t)£ cO 



(3-24) 



* 

To place the maximum restrictions on f ( t ) and the least restrictions 

on f(t), the associated cC and K are obtained from the line tangent to 

the modified polar plot of G(jw) at the real axis intercept of the plot. 

As required by Theorem Three, the plot must lie on or to the right of 

this line. The slope of this line determines the maximum value of cC , 

» 

and thus the restrictions on f (t) , and the maximum value of K. Equa- 
tions (3-18) and (3-19), for this choice of cC and K, reduce to 

0 £ f(t)4cO (3-25) 

f(t) £ 2pf(t) (3-26) 

The lines representing these two extremes are shown in Figure 3-5. In 
between these two lines is a compromise line (dotted). This line 
represents the tradeoff between the two sets of restrictions. The 
value of oC and K to determine the restrictions on f(t) and f(t) are 
obtained from the slope and real axis intercept of the compromise line 
which can vary between the two extremes. The following steps illustrate 
the application of Theorem Three. 

(a) Plot the modified polar plot of G( jw) . 

(b) From the plot determine whether or p should be zero. 

(c) Construct the line representing the maximum restrictions 
on f(t) by constructing a vertical line tangent to the 
modified polar plot of G(jw) such that the plot lies to 
the right of the vertical line. 

(d) Construct the line representing the maximum restrictions 
on f(t) by constructing a line tangent to the modified 
polar plot of G(jw) at the real axis intercept such that 
the plot lies to the right of the line. 
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(e) All the possible compromises lie in between the two ext fie me 
lines* Any compromise line is valid provided the modified 
polar pLot ot 0( jw) lies to the fight of It* 

(f) The values of tC and K are obtained from the slopes and 
real axis intercepts of these lines i 

In the event that the time varying gain in the feedback loop has an 
f(t) that is not bounded, one must utilize Theorem TWO to obtain the 
range of Values of f(t) for which the system is stable* brocket! has 
developed a stability criterion which is identical to Theorem Two but 
Is easier to apply because it uses the ordinary Nyquist plot* 

12 

3*4 brocke t t s Circle Criterion * The circle criterion establishes 
the range of values of f(t) which will assure stability assuming UO 
restrictions on f(t)« The significance of this criterion is twofold* 

It is easy to apply and it reduces to the Nyquist ciiteiidil as a special 
case* The circle criterion is contained within the following theOfemf 

Theorem Four Let q(s) and p(s) be polynomials Without Common fUetOiS 



be an irreducible representation of G(s) = q(s)/p(s) (system i§ 060= 
trollable and observable)* Then, if p(s) has no £ef6s in the fight 
half plane 

(a) Ail solutions of Equation (3-2?) are bounded if 



and le t 



X = A X 4- fc /U 

u = c r x 

y = c T x 



(s-an 



(3=§fi) 

( 2 * 29 ) 



0 4 (is .4: f(t) 4 0^ 
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and the Nyquist locus of G(s) does not encircle or inter- 
sect the open disk which is centered on the negative real 
axis of the G(s) plane and has as a diameter the segment 



of the negative real axis ( - 



) 



5 £ 

(b) All solutions are bounded and go to zero at an exponential 
rate if there is some £>Osuch that 






(3-31) 



and the Nyquist locus behaves as in (a). 

This theorem is also concerned with the time varying system of Figure 
3-1. The open disk is shown in Figure 3-6. Part (a) of the criterion 
simply says that if the Nyquist plot of G(s) does not intersect or en- 
circle the disk of radius 1/2 ( - — p- ) and center at -3/2<£ + 1/2 

then the system is stable in the sense of Lyapunov. Note that if the 
lower limit on f (t) , ^ , is zero then the disk becomes the entire 

plane to the left of -1/22. This is the same result as was obtained 
using Theorem Two. To show that this criterion reduces to the Nyquist 
criterion, consider f(t) to be a constant K. Then the open disk reduces 
to a point at -1/K. The system is stable, provided the Nyquist plot 
of G(s) does not intersect or encircle the point -l/K. But this is 
simply the Nyquist criterion. Part (b) of the criterion implies that 
if one constructs the largest disk possible that is not intersected or 
encircled by the Nyquist plot G(jw) and can find an £ y Q that will trans- 
late the oC and associated with the largest disk to the actual <£ 

and (3 of f(t), then all solutions go to zero at an exponential rate. 
This is asymptotic stability and not just bounded output stability. 

To demonstrate the use of the circle criterion consider the following 
example : 
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Example 3.2 The open loop plant of Figure 3-1 has the transfer 
function 

G(s) = S( S+2) (3-32) 

and the Nyquist plot is shown in Figure 3-7. The time varying gain has 
the following limits: 

0.1 != f(t) ^ 2 (3-33) 

and the circle criterion disk for this f ( t ) is also plotted in Figure 
3-7. Since G(jw) does not intersect or encircle this disk, the system 
is stable. In fact, the system is stable for the following range of 
values for f(t): 

0 £ f(t) ^ 4 (3-34) 

These limits are established by constructing the largest disk subject 
to the requirements of the theorem. This circle is the entire plane 
to the left of line 

min Re G(jw) = -0.25 (3-35) 

This line corresponds to an &C of value 4. It is obvious that the 
system is not asymptotically stable because there is no £ > O that 
will translate the and of the infinite circle to the actual^ 

and ^3 of f (t) . 

The previous sections of this chapter have considered the problem 
of stability for time varying systems. The various stability criteria 
for these systems were developed primarily from the stability theorems 
of Lyapunov’s second method. At first glance it might appear that 
the design of controls systems by the optimization technique eliminates 
the stability problem. This is not true. Kalman, in Reference 3, 
makes this clear by the statement, "Optimality does not imply stability!" 
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One might also question the relation between non-linear (and time 
varying) systems and the design of linear systems by the optimal control 
theory. The answer to this question is that the optimal control 
approach, more often than not, leads to a non-linear or time varying 
system. The next section considers the problem of stability as it 
relates to optimal control. 

3.5 Stability and Optimal Controls . Lyapunov stability theory is 
very similar to optimal control theory in many respects. The formula- 
tion of the optimization problem consists essentially of selecting the 
cost function that expresses the desired objective. This function may 
be dependent upon the system states, time, the control, or all three. 

The choice of the index is not an easy one to make. However, assuming 
that the choice has been made, this function can usually be translated 
into the state space as a family of constant cost surfaces. The 
solution to the optimization problem is the control that produces 
system state trajectories that transit these surfaces in a manner such 
that they are always headed toward a lesser cost surface. The deter- 
mination of stability consists of a search for a Lyapunov function 
which may be dependent upon the system states and time. If this function 
and its time derivative behave in a prescribed manner, then stability 
is assured. The Lyapunov function also may be translated into the 
state space as a family of constant value surfaces. If the unaltered 
system state trajectories transit these surfaces in a manner such that 
the Lyapunov is a decreasing function of time, then stability is assured. 
But this is where the similarity ends. As emphasized by Kalman, the 
optimal control approach may lead to an unstable system unless some 
sort of stability consideration is incorporated into the optimization 
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problem. The obvious place to inject this consideration is in the 
choice of performance index. That is, to select a performance index 



that is a valid Lyapunov function for the optimal system . For 
instance, if the integral squared error criterion is chosen for the 
free ( JL( - 0) linear regulator then this criterion is a Lyapunov 
function for the optimal system provided the squared error criterion 
does not vanish along any state trajectory of the optimal system. 
This result is crystalized in the following theorem by Kalman. 

Theorem Five ^ Consider a free ( JU = 0) , linear, time invariant 
dynamic system with an equilibrium state at the origin and assume 
(a) The error criterion [_ (x) is positive definite and 



Then the equilibrium state is asymptotically stable. 

This theorem is applicable to the parameter optimization problem, 
which selects the value of a system parameter that minimizes the per- 
formance index. Stability of this type of system is assured by placing 
certain restrictions on the cost function that requires it to be a 
valid Lyapunov function for the optimal system. It is interesting to 
note that the error criterion does not have to be quadratic. The 
positive definite requirement on [_ (x) ensures that the cost function 
or the Lyapunov function does not vanish anywhere other than at the 
origin. If it did vanish, the optimal system would have an equilibrium 
state other than the origin. This is the reason why the Q and P 



L(o) = o. 



(b) The performance index 




(3-36) 



o 



is finite in some neighborhood of the origin. 
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matrices of the performance index of Chapter II were required to be 
positive semi-definite and positive definite, respectively. 

The above theorem is applicable to the unforced linear regulator 
but not to the true optimization of the forced linear regulator for 
which the optimal control is derived. This problem may have a finite 
control time and thus a new stability criteria is required. Kalman 
has considered this problem and with the following results. 

3 

Theorem Six . If the assumptions 

T T 

(a) The pair (A, L ), where Q = LL and A is the A matrix of 
the plant state equations, is observable. 

(b) The plant is controllable 

are satisfied then R(t =eO) is positive definite and the optimal control 
law is stable. 

The first assumption of this theorem ensures that the integrand 
of the cost function 

X T QX + U T PU (3-37) 

does not vanish along the optimal trajectories. This implies that the 
cost function is a Lyapunov function for the optimal system. Since 
the evaluated cost function is 

J = X T R X (3-38) 

then R must be positive definite. The second assumption guarantees a 
solution, R, to the matrix Ricatti differential equation. 

This chapter has considered the problem of stability for time 
varying systems and the problem of stability for optimal systems. The 
similarity of stability theory and optimal theory suggests that optimal 
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control theory might be useful in establishing stability criteria for 
non-linear and time varying systems. The linear regulator problem 
with finite control time suggests such an idea. For that particular 
problem the feedback gains are time varying but stability is still 
assured if the requirements of Theorem Six are met. The optimal controls 
approach would be useful not for the purpose of designing an optimal 
system but for its ability to yield non-linear and time varying systems. 
The optimal approach might provide a refreshing look at the stability 
of these systems. 
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Figure 3-1 • Block Diagram of Time Varying System 



PCs) 




R= Mi) 



Figure 3-2. Network Equivalent of Figure 3-1 



uj Im (*rCj' A ^ 



Re (aCjw) 



Figure 3-3. Axes of Modified Polar Plot 
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Figure 3-4. Modified Polar Plot 
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Figure 3-6. Circle Criterion Disk 




Figure 3-7. 



Circle Criterion 



for Example 



3.2 



Re G L]w) 



Re G 
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APPENDIX A 



A. I The following is a list of the state equations of the networks 
and machines of Figure A-I through A-7 respectively. 

(1) Phase Lag Network 

Vc - feflVc +[Rc]e fA (A-l) 

= [l ]v C (A- 2) 

(2) Phase Lag Network with High Frequency Attenuation 



Vc=L C(R,4rJ\/c -f- [cCR. + rJe.v, 



,= [ R, 4 R Jv c -f L R.+ R, ] 



Eh 



(3) Phase Lead Network 



Vc - [ rc] Vc 4- [rc]e 






= ['lJVc + [i]&. 



(4) Phase Lead Network with Fixed DC Attenuation 



p r. + r* i pi_ i 
Vc = L R, RiC JVc + L R-a- cj E In 



(A-3) 



(A-4) 



(A-5) 



(A-6) 



(A-7) 
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(A-8) 



u - Emvc +- D i t !n 



(5) DC Motor with Constant Field Current and Negligible 
Armature Inductance 



o 

3 

i 




0m 





O 

o 



Kf Km 

jrJ. 



©m 

6 m 






o 

Kt 

JFU_ 






(A-9) 



Or 



- [ I o ] 



0m 

! 0rvi 



(6) DC Motor with Constant Armature Current 



(A-10) 



If 

©m 



Igf 



Kl 

J 



o 



-I i 

J J 



If 

o m 



-h 



Lf 

O 



t ir\ 



(A-ll) 





— 


[o 


1 


V 










> 

1 


(7) 


Two 


Phase Servomotor 






0m 




o 


1 


1 


1 


• • 

On 




0 


-(MO 

J 


a 



(A-12) 



■f 



o 



. J J 



(A-13) 



©m = 



O 



Om 



(A-14) 



€t 
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The following is a list defining the constants and variables of the 
above state equations. 



Km 

Kt 

Ra 

Rf 

Lf 

la 

If 

Ein 

Om 

T 

Jm 

J L 

J 

fm 

f L 

f 

Ke 

Kn 



motor back emf constant 
motor torque constant 
armature resistance 
field resistance 
armature inductance 
armature current 
field current 

applied armature or field voltage 

motor shaft angular position 

gear box ratio = 

motor inertia 

load inertia 

Jm + T 2 J l 

motor friction 

load friction 

fm + T 2 f_ 

Li 

motor torque constant 

motor vicous friction constant 
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Figure A-l. Phase Lag Network 




Figure A- 2. Phase Lag Network with High Frequency 
Attenuation 



Vc 





Figure A-4. Phase Lead Network with Fixed DC 
Attenuation 
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Figure A-5* DC Motor with Constant Field Current 
and Negligible Armature Inductance 




Figure A-6. DC Motor with Constant Armature Current 



Ref Field 




Figure A-7. Two Phase Servo Motor 
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APPENDIX B 



B.l. Derivation of the L Matrix for a Second Order System 





X,X, dt 





From Equation (2-52) the L matrix is 




B . 2. Derivation of the L Matrix for a Third Order System 



(B-l) 



(B-2) 



(B-3) 



(B-4) 



(B-5) 




+ ^22 X, +^33 / , 



(B-6) 



t?0 

(b-7) 

O 

*0 

x.x, i- x, x, - Jfilpl x, x, D d t 

o 
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(13-8) 



oo 





K,(o) t 




X,X, dt 



j x, t Sp? X|+\l^s X,] + Z ^ i J 



© 




(n*9) 




From Equation (2-52) the L matrix ie 




(B-ll) 
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APPENDIX C 



C . 1 Mathematical Formulation of the Ricatti Differential Equation 

for a Third Order Plant , The state equations for the plant of 
Figure 2-5 are 







"o 


1 


0 


V 




'O' 


Xz 


— 


0 


0 


1 




+ 


0 


_x 3 . 




o 


-ab 


-(4+b) 


x 3 




L 



u 



De f ine 



F = a + b 
G = ab 



(c-l) 



(C-2) 

(C-3) 



The matrices required by the Ricatti equation are 





0 


1 


0 




A = 


0 


0 


1 






0 


-G 


-F 






r - 1 
»— 1 


/?12 


X 13 


R = 


^12 


/? 22 


^23 




^13 


X 23 


^33 



G 

P = P U 



(C-4) 



(C-5) 



(C-6) 



(C-7) 
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Q = 



q n 0 



' 1 . 



22 



0 

0 






The terms of the Rlcatti differential equation 



A*R + RA - RBP'Vr + Q = -R 



are 

o o o 

f\ R - 



( 0 - 8 ) 



(0=9) 



( 0 * 10 ) 



RA - 



0 4 // ' 



O A. rt“ G ft l% ft ix~ ^ ^ 









O Shi-tjAii /l 1 i‘* Mis 



“2 ^ 

- 6 /?\ -&n»/Ui -<* n n /i 

p" 



-RBPBR = 



P" 



P" 



2 5 



P* P M pr, 

* ^ 1 



... - - 4 

p" p* p.» 



( 0 = 11 ) 



(0-12) 



Substituting the above terms into the Ricatti equation yields the 
following six first order non-linear differential equations j 



A A i 



"4. Af%(0 + fr* £ * A/, it) 

S>.. 0 



(0*13) 



- a ^/UiU)t An it) ft* tt)= * 4 a U) 

pH 



(C*U) 



(C-15) 



/lib(t) + /In Ci) ~ 'FA.nH') = _ A>iCt) 

P" 



-6. /IhM +^ii + z(/b*(.0- 6r4iitt)) - - /hilt) (C-16) 



4_ i /l&L-t)-!* Asi(t) iAiz ({) ~ fAziii) - ~ A^^t) (C-17) 

P" 



- 6 ! 

p- 



AllL't') + ^35 



+• Z (/liitt) - ^ /LiittSj 



- AaAt) 



(C-18) 



Note that the number of first order equations which must be integrated 
to obtain the solution to the Ricatti equation is 



n (nfl) 
2 



(C-19) 



where n is the order of the plant. These equations are easily solved 
on a digital computer by integrating backward in time with the following 
initial conditions: 

R (t = o) = 0 (C-20) 



The solution to these equations is easily checked by using Equation 
(C-13). The steady state value of /? ^ is 



A 



13 




(C-21) 



For higher order plants. Equation (C-21) becomes 



Ain 




(C-22) 



This result is valid if the description of the plant is of Normal Form. 
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APPENDIX D 



D.l. Block Diagram Manipulations . When control systems are represented 
by block diagrams, manipulations are often performed to convert the 
system to a convenient form for analysis. Manipulations such as these 
often result in a new set of state variables for the system. Thus, a 
block diagram manipulation is comparable with the changing from one state 
variable representation to another. For the linear regulator problem, 
the Normal Form has an advantage over the others because of the simple 
modeling procedure due to Schultz and Melsa. Whatever representation 
is selected, one must be aware of the effect of these changes on 
optimization process. The problem of interest is again the second order 
linear regulator. In Chapter II this problem was solved by substituting 
the A matrix of the plant of Figure 2-1. Since there is a certain 
amount of matrix algebra that must be performed to obtain the solution 
to the Ricatti equation, the problem is simplified by moving the pole 
of the plant into the feedback loop. This is a valid operation since 
it is known that the optimal control is state variable feedback. 

Figure D-l shows this manipulation. The optimal feedback gains with- 
out this manipulation are represented by and K^. Now, the open 
loop plant of Figure D-l is optimized by the same procedure used in 
Chapter II. The optimal k^ and k,, for the new plant are obtained and 
then and are found from the relations 

K x = k x (D-l) 

K 2 = k 2 - a/G (D-2) 

and K 2 are not the same as the K ( and K 2 obtained in Chapter IT, 
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This is reasonable since the optimization process does not take into 
account the ability of the actual plant to drive the state vector to 
the origin. However, the same optimal system will result if the Q and 
P matrices are also changed for the new plant as follows: 



The bar indicates the new plant, and the hat indicates the original 



the state as compared to emphasis on minimizing the control for state 



of state did not change. One might look at this result in the light 
that, since the pole is now used as control, the emphasis on minimizing 
the state must be increased over the emphasis on minimizing the control 
to obtain the same optimal system. 

The very same procedure is carried out for the case where both 
the pole and the open loop gain are moved into the feedback loop. 

Figure D-2 shows this manipulation. For this case, the Q and P matrices 
must be changed as follows: 




(D-3) 




A 



(D-4) 



plant. Note that the ^22^11’ represents emphasis on minimizing 



X^ 9 increases as the square of the quantity added to the feedback loop 
of state The term ^id not °hange because the feedback loop 




pn pn 



(D-5) 




(D-6) 
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These equations are CI^ times the results obtained lor the case where 
only the pole is moved. Thus, the same optimal system is obtained by 
moving all poles and open loop gains Into the feedback loops and then 
increasing the appropriate by the square of the pole values 

and then multiplying this result by the square of the open loop gain. 
This result is useful because it introduces the maximum number of zero 
elements into the A matrix. It also demonstrates the effect of block 
diagram manipulations on the optimization problem. 




Figure D-l. Optimal System with Pole Moved 
into Feedback Loop 




Figure D-2. Optimal System with Pole and Gain 
Moved into Feedback Loop 
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APPENDIX E 



E . 1 Derivation of the System, State Equat i ons for Complex tntcr - 

connec t ions . In Chapter I it was stated that the combined state 
equations could be rearranged in the following manners 



^ : Ax + f B b a 6 <E-i) 

■*= C&d- + D b tfi> (E-2) 

/ - A ^ (E*3) 

Kb is the vector of component inputs that are not system inputs. 
Solving Equation ( E - 3) for Y and substituting into Equation (E*2) 
yie Ids 

£/lb - C* 4- 4- Db4i (E-4) 

Solving this equation for r^ yields 

/n = [a OkTc& h- O' 1 - Dbl'o. 4* (e- 5) 



Substituting this result into Equation (E“l) yields 



X-- Wi-BfeU' 1 * 



Dbj'cj 6 



4 S&*4Bb[£' DbjCVj g* (e-6) 






* The term 




(E-7> 



Aw 
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can be written as 



-I -i 



[A -1 - Db] A A 



or 



^A[A"'-Db]j L 



A 



But, this expression reduces to 



[I-ADb] & 



(E-8) 



(E-9) 



(E-10) 



Therefore, Equation (E-6) reduces to 

-AdJ A cj x + ^e>cL+ e>b[l-ADbl AD^ /2* (E-ID 

The output equations, Y, are obtained by substituting Equation (E-5) 
into Equation (E-2) and rearranging. The result is 




V = ^C4DbCl-ADblAcjA + + DbLl- ADQA0a| A, 



(E-12) 



The existence of the inverse of (X-ADb) i s assured by consider- 
ing a linear time -invariant system. If (X-ADb) does not exist, 
then the system does not have a characteristic equation. But, every 
linear system has a characteristic equation. 
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